Analysis of self-describing gridded geoscience data with netCDF Operators (NCO)
نویسنده
چکیده
The netCDF Operator (NCO) software facilitates manipulation and analysis of gridded 2 geoscience data stored in the self-describing netCDF format. NCO is optimized to efficiently 3 analyze large multi-dimensional datasets spanning many files. Researchers and data centers 4 often use NCO to analyze and serve observed and modeled geoscience data including satel5 lite observations and weather, air quality, and climate forecasts. NCO’s functionality includes 6 shared memory threading, a message-passing interface, network transparency, and an inter7 preted language parser. NCO treats data files as a high level data type whose contents may 8 be simultaneously manipulated by a single command. Institutions and data portals often use 9 NCO for middleware, to hyperslab and aggregate dataset requests, while scientific researchers 10 use NCO to perform three general functions: arithmetic operations, data permutation and com11 pression, and metadata editing. We describe NCO’s design philosophy and primary features, 12 illustrate techniques to solve common geoscience and environmental data analysis problems, 13 and suggest ways to design gridded datasets that can ease their subsequent analysis. 14
منابع مشابه
Scaling Properties of Common Statistical Operators for Gridded Datasets
An accurate cost-model that accounts for dataset size and structure can help optimize geoscience data analysis. We develop and apply a computational model to estimate data analysis costs for arithmetic operations on gridded datasets typical of satelliteor climate model-origin. For these dataset geometries our model predicts data reduction scalings that agree with measurements of widely-used geo...
متن کاملNetCDF: A Public-Domain-Software Solution to Data-Access Problems for Numerical Modelers
Unidata's network Common Data Form, netCDF, provides users with an e cient set of software for scienti c-data storage, retrieval, and manipulation. The netCDF le format is machine-independent, direct-access, self-describing, and in the public domain, thereby alleviating many problems associated with accessing output from large hydrodynamic models. NetCDF has programming interfaces in both the F...
متن کاملMerging the NetCDF and HDF5 Libraries to Achieve Gains in Performance and Interoperability
This proposal seeks to advance the technology available for accessing scientific data and for integrating observations with model representations in multiple disciplines by extending and merging the Network Common Data Form (netCDF [8]) library and the Hierarchical Data Format-5 (HDF5 [10]) library. These two independently developed data-access libraries and formats provide simple, efficient, a...
متن کاملImplementing a Parallel NetCDF Interface for Seamless Remote I/O Using Multi-dimensional Data
Parallel netCDF supports parallel I/O operations for a view of data as a collection of self-describing, portable, and array-oriented objects that can be accessed through a simple interface. Its parallel I/O operations are realized with the help of an MPI-I/O library. However, such the operations are not available in remote I/O operations. So, a remote I/O mechanism of a Stampi library was intro...
متن کاملTechnical note: Recursive rediscretisation of geo-scientific data in the Modular Earth Submodel System (MESSy)
This technical note describes a method for the rediscretisation of “gridded” geo-scientific data. A recursive algorithm (NREGRID) is derived to solve the rediscretisation problem for orthogonal grids (including curvilinear) of arbitrary dimension. The algorithm is used within the program NCREGRID to handle geo-scientific data. These data are typically 2-dimensional (latitude-longitude grid), or...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- Environmental Modelling and Software
دوره 23 شماره
صفحات -
تاریخ انتشار 2008